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Abstract 



The No Child Left Behind Act of 2001 (NCLB) requires that the nation’s public elementary and 
secondary school systems be held accountable for achieving high levels of educational 
proficiency for all students. This law represents an unparalleled extension of the federal role into 
the realm of local educational accountability. Specifically, NCLB mandates that all states 
establish performance-based accoimtability systems that include: clear standards and goals for 
improvement; rigorous methods of measuring progress towards established performance targets; 
and high-stakes consequences for schools and districts that fail to make sufficient progress in 
reaching the goal of universal student proficiency. While achievement testing will be the central 
component of these state accountability systems, high school graduation rates are also a 
requirement indicator of performance at the secondary level. This report seeks to inform the on- 
going debate over high school graduation rates with particular attention to the ways in which No 
Child Left Behind has effectively both redirected attention toward graduation rates and reshaped 
the contours of that debate. We begin by briefly introducing the provisions of NCLB that pertain 
to high school graduation and discussing their implications from a measurement perspective. 
Next we present several distinctive strategies for developing a high school graduation indicator 
that are broadly consistent with the new federal requirements for accountability. The advantages 
and disadvantages of each strategy are addressed. In the empirical portion of this study we 
construct these proposed high school graduation indicators using information from the Common 
Core of Data, the U.S. Department of Education’s census of local educational agencies and 
schools. These indicators are created at the district level for the high school class of 2000, the 
most recent year for which complete data are available. We then systematically compare the 
results generated by the respective measures, with particular attention to: their point estimates 
and distributions; the quality of the indicators from an informational perspective (e.g., coverage 
of school districts or student population); and differential perfonnance of the indicators when 
disaggregated by state and by agency characteristics such as minority enrollment and district 
size. In each of these areas, analyses reveal substantial differences among the three alternative 
graduation rate indicators examined. In particular, two troubling findings emerge regarding 
graduation indicators that rely heavily on information about high school dropout. First, these 
indicators cannot be calculated for many states and districts due to missing information on the 
number of students who drop out of high school. Second, in cases where values can be 
computed, dropout-dependent measures appear to dramatically overestimate graduation rates 
compared to alternative indicators. We conclude by discussing several lessons this study may 
offer for future research and the policy implications for measuring graduation rates under 
conditions of high-stakes accountability. 
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1. Introduction 

The relationship between levels of educational attainment and individual economic and 
social outcomes has been well documented. Individuals with higher levels of education (and 
more advanced credentials) enjoy higher income, more stable employment, and less dependency 
on public assistance. The more well-educated are also less likely to experience a variety of 
detrimental social outcomes, including early childbearing, reports of ill health, incarceration, or 
criminal victimization. Until about the 1970s, a high school diploma was generally viewed as a 
credential that would ensure a reasonably secure and well-paying job. But with increasing 
regularity, discussions of the returns to education have focused on attaining a college education 
as a prerequisite for economic success in today’s labor market. This shift of attention is largely 
justifiable. Rates of college matriculation have increased considerably over the past 20 years, as 
have the economic returns to a bachelor’s degree, often termed the “college premium” (Murphy 
& Welch 1989; Juhn & Murphy 1995). As a larger segment of the population completes college 
and enters the workforce with a postsecondary degree, it is reasonable to anticipate that the 
college degree will supplant a high school diploma with regard to hiring decisions and also as a 
more generalized indicator of educational and social status. 

In light of the ascendancy of the college credential, there may be a temptation to view the 
high school diploma as becoming economically and socially obsolete. On the other hand, 
however, it is imperative that we not lose sight of the relevance that a high school education 
continues to hold for young adults, even today. Much like the college premium noted above, 
high school graduates consistently outperform non-graduates on a variety of social and economic 
indicators, often by substantial margins (Chaplin & Lerman 1 997). Obtaining a high school 
diploma can also represent a critical first step in gaining access to a college education and. 
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eventually, earning the bachelor’s degree that will further enhance an individual’s prospects for 
advancement in an increasingly competitive society. 

One of the reasons that the issue of high school completion has garnered relatively minor 
attention in recent years could be the common perception that high school dropout rates have 
historically been on the decline and have stabilized at rather low levels during the past decade. 

In fact, widely- reported estimates based on results from the Current Population Survey (CPS) 
place the dropout rate among 16-24 year olds at about 1 1 percent nationwide in 2000, a level that 
some might view as nearing the limit of the high school dropout rate practically attainable 
(Kaufman, Alt & Chapman 2001). Despite this appearance of a growing consensus, a dissenting 
perspective has arisen from a body of research that has challenged the accuracy and reliability of 
commonly-reported statistics on high school completion and dropout (see Chaplin 2002, Greene 
2002a). These studies cite several factors that tend to systematically inflate the high school 
completion rates commonly reported by the U.S. Department of Education, Census Bureau, and 
other authoritative sources, particularly when these estimates are based on survey data, as is the 
case for the CPS.' These potential sources of bias include: the frequent decision to count GEDs 
along with regular diploma recipients as high school graduates, sample undercoverage for certain 
segments of the population (e.g., those incarcerated), and self-reporting bias regarding levels of 
educational attainment. 



* The CPS has provided the backbone of the annual dropout report of the National Center of Education statistics 
since 1988. For many decades, it was the only source of nationally representative data available. 

^ The issue of which credentials should be included in estimates of graduation rates is salient for several reasons. 
First, a great deal of evidence suggests that GED recipients do not fare nearly as well as regular graduates in terms 
of labor market and educational outcomes (Cameron & Heckman, 1993; Mumane, Willet & Tyler, 1998; and 
Boesel, A1 Salam & Smith, 1998). In addition, different treatment of the GED credential is a major source of 
discrepancies in reported estimates of high school completion rates. Indicators that include GEDs produce far more 
positive results (i.e., higher completion rates) than more conservative estimates that exclude the GED (Chaplin, 
2002). For example, completion rates that count GED recipients among high school graduates have remained stable 
or risen since the 1970s, while rates excluding the GED are much lower on average and have fallen over the same 
period of time (Chaplin, 2002; Cameron & Heckman, 1993; Kaufman, 2000). 
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Just as high school completion arguably remains an important predictor of an individual’s 
future economic success, it also represents a key indicator of performance for educational 
systems. In addition to producing higher- achieving students, effective schools, districts, or states 
are expected to retain a greater proportion of students until completion of secondary schooling. 
Consequently, such educational units should strive to achieve higher graduation rates. Public 
and scholarly attention to high school dropout and completion has been rejuvenated in recent 
years as handfuls of states and large districts around the country have introduced new 
educational accountability systems. This interest has been largely motivated by the fear that 
imposing high stakes testing, exit exams for graduation, and policies ending social promotion 
would create pressure for low-performing student to exit (or be removed from) the system and 
produce increased dropout rates (Lillard & DeCicca 2001; Herbert & Hauser 1999; Bonsteel & 
Rumberger 1999; Haney 2000). In addition, the trend toward more systematic forms of 
accountability on a national scale further raises the stakes for measuring high school completion 
accurately and consistently. Specifically, new federal educational legislation authorized in the 
No Child Left Behind Act requires that all high schools explicitly take completion rates into 
account (along with achievement test scores) when measuring their current level of performance 
and the progress they have made towards reaching their long-term performance goals. 

This paper seeks to inform the debate over high school graduation rates with particular 
attention to the ways in which No Child Left Behind has effectively both redirected attention 
toward graduation rates and reshaped the contours of that debate. Since one of our main interests 
in this study is considering the impacts of these federal policies, we focus throughout the paper . 
strictly on the public education system. Although subject to performance pressures from market 
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forces, private schools are not directly subject to the performance-based accountability 
requirements of NCLB or other federal statutes. We begin by briefly introducing the provisions 
of NCLB that pertain to high school graduation and discuss their implications from a 
measurement perspective. Next we present several distinctive strategies for developing a high 
school graduation indicator that are broadly consistent with the new federal requirements for 
accountability. The advantages and disadvantages of each strategy are addressed. In the 
empirical portion of this study we construct these proposed high school graduation indicators 
using information from the Common Core of Data, the U.S. Department of Education’s census 
of local educational a^ncies and schools. These indicators are created at the district level for 
the high school class of 2000, the most recent year for which complete data are available. We 
then systematically compare the results generated by the respective measures, with particular 
attention to: their point estimates and distributions; the quality of the indicators from an 
informational perspective (e.g., coverage of school districts or student population); and 
differential performance of the indicators when disaggregated by state and by agency 
characteristics such as minority enrollment and district size. We conclude by discussing several 
lessons this study may offer for future research and the policy implications for measuring 
graduation rates under conditions of high-stakes accountability. 

2. No Child Left B ehevd, Accountability, and High School Graduation 

The No Child Left Behind Act (NCLB), written into law in January 2002, is the most 
recent reauthorization of the Elementary and Secondary Education Act (ESEA), the federal 
government’s omnibus legislation for public education at the elementary and secondary levels. 
The provisions of NCLB strongly reflect the (George W.) Bush administration’s emphasis on 
raising standards for educational performance and accountability, combined with increased 
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flexibility over the disposition of federal funding at the state and local levels. Among the most 
ambitious and controversial elements of NCLB stands the requirement that each state will 
develop a comprehensive plan detailing a strategy by which it will (1) ensure that every student 
attains educational proficiency and (2) elim in ate achievement gaps between high and low 
performing groups within 12 years (i.e., by the 2013-14 academic year). Although the specifics 
of defining and implementing certain key elements of their standards and accountability system 
remain in the purview of the states themselves, their plans must conform to the terms of the 
federal legislation. 

Similar goals have been incorporated into past federal legislation, but NCLB exceeds 
these earlier attempts to raise standards in its specificity and the high stakes attached to failure to 
meet established goals. For instance, state plans must identify concrete annual performance 
targets or milestones for attaining their long-term goals — that is, they must meet an established 
definition of adequate yearly progress, or AYP. Failure to meet performance goals for AYP in 
successive years will result in the imposition of progressively more severe sanctions. These 
include: public dissemination of annual school report cards and identification of schools not 
meeting AYP goals as “in need of improvement”; restrictions on the use of certain federal funds; 
making available to students in failing schools the option of transferring to another public school 
not identified as in need of improvement (i.e., public school choice); offering students from low- 
income families the option of using Title I funds to secure supplemental educational services 
(e.g., tutoring) from an approved public- or private- sector provider; and additional sanctions that 
might include changes to staffing, administration, or curriculum and eventually more 
fundamental school restructuring. These standards of performance-based accountability must be 
established for states, districts, and schools. Within each of these units, goals for AYP must be 




5 



10 



Swanson & Chaplin 



Counting High School Graduates 



met separately for specific segments of the student population defined on the basis of race and 
ethnicity, socioeconomic status, disability, and level of English language proficiency. 

Large-scale student assessments represent the cornerstone of NCLB accountability and 
are a required component for defining AYP at all levels of the public elementary and secondary 
education system. For instance, the law requires that by the 2005-06 academic year all students 
statewide will be tested in mathematics and reading annually in grades 3-8 and at least once in 
grades 10-12. States must also expand the scope of their assessment program over time by 
introducing testing in additional subjects at additional grade levels, as specified in the legislation. 
In addition, the material included in these assessments must be aligned with the state’s subject- 
specific academic content standards. States were required to submit their performance-based 
accountability plans and definitions of AYP no later than January 3 1 , 2003. As of 2002, nearly 
all states had academic content standards in most core subjects and mandated statewide testing 
programs already in place (Editorial Projects in Education 2003). However, in many cases state 
accountability systems may not be in compliance with specific aspects of federal requirements 
under NCLB, such as standards-asscssment aligiunent, grade levels or subjects tested, and the 
ability to report disaggregated achievement results for specific demographic groups. 

Given the technical complexity and expense of instituting and maintaining a large-scale 
student testing system that meets the standards of NCLB, a tremendous amount of time, energy, 
and effort have been devoted to the politics and mechanics of student testing. By comparison, 
very little attention has been afforded in public forums to NCLB’s requirements with regard to 
high school graduation. Although student test scores must be the primary performance measure 
at all educational levels, definitions of AYP must incorporate at least one other indicator of 
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academic performance. At the secondary level, this definition must include graduation rates. 

Specifically, the legislation states that the definition of AYP for high schools: 

“. . . includes graduation rates for public secondary school students (defined as the 
percentage of students who graduate from secondary school with a regular diploma in the 
standard number of years)” [Sec 111 l(b)(2)(C)(vi)]. 

All indicators of academic performance employed in state accountability system (including 

achievement test scores, graduation rates, and other measures) are required to meet acceptable 

standards of statistical validity and reliability. States must: 

“. . . ensure that the indicators described in those provisions [defining Adequate Yearly 
Progress] are valid and reliable, and are consistent with relevant, nationally recognized 
professional and technical standards, if any” [Sec 111 l(b)(2)(D)(i)]. 

Additional guidance provided in the final Title I regulations offers further clarification on several 

specific points that bear directly on methods for calculating these graduate rates for purposes of 

accoimtability imder NCLB. First, graduates are considered to be only those students to receive 

a regular high school diploma, which must be fully aligned with the state’s academic content 

standards. Therefore students receiving a GED or other state- issued credential (such as an 

attendance certificate or other form of recognition) that fails to meet those standards will not be 

coimted as a graduate. Final guidance also explicitly stresses that states must avoid classifying 

dropouts as transfer students for purposes of calculating the high school graduation rate. 

Although this provides more explicit guidelines for calculating graduation rates, the 

federal regulations do allow states some degree of flexibility. State plans may develop 

definitions of AYP that employ: 

“another definition [of high school graduation rate], developed by the State and approved 
by the Secretary in the State plan, that more accurately measures the rate of students who 
graduate from high school with a regular diploma” [Federal Register Vol. 67, No. 231]. 
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In addition, the final regulations declined to provide explicit guidance on the important issue of 
whether graduation rates must be based on a longitudinal data from an accountability system that 
tracks individual students over time. Under these circumstances, therefore, states are presumably 
free to develop high school graduation indicators based on cross-sectional data from student 
cohorts. 

3. Measurement Considerations 

In comparison to the psychometric intricacies of measuring student achievement, the 
challenges associated with counting the number of students who complete high school in a given 
school or district may at first appear trivial. Here we are attempting to measure a very clear and 
(in theory) observable outcome — whether or not a student graduates from high school. 

However, the endeavor of calculating high school completion rates with an acceptable degree of 
validity and reliability is not without its own challenges and subtleties. In certain respects, the 
difficulties associated with measuring completion bear a striking resemblance to issues 
encountered in assessing student achievement. Although a comprehensive treatment of all the 
conceptual and technical considerations involved in the measurement of graduation rates is 
beyond the scope of the current paper, we briefly touch upon several key issues that bear directly 
on the development of valid indicators for high school graduation under the accountability 
regime established by NCLB. Interested readers are directed to the works cited for more detailed 
discussions of these and related topics. 



^ In fact, among the five states receiving early approval of their state accountability plans only one (Colorado) has 
adopted a method for calculating high school completion rates that explicitly specifies the use of longitudinal 
student data. Several other states, however, suggest that longitudinal measures may be phased in over time. 
Approved state accountability plans have been publicly posted by the U.S. Department of Education (see 
http://www.ed.gov/offices/OESE/CFP/csas/index.html). 
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The field of psychometrics, which deals with the measurement of cognitive performance, 
seeks to obtain an imperfect but observed estimate (i.e., test score) of a more fundamental but 
unobservable construct of interest (i.e., academic achievement). Although some methods of 
measuring achievement may be more or less valid and reliable, there is no single, universally- 
accepted way to measure this or most other related constructs. That is, it is possible to design 
variety of different instruments (i.e., tests), which, from theoretical and psychometric points of 
view, each adequately capture the same underlying achievement construct. Despite the obvious 
differences between achievement and completion outcomes, the task of measuring the 
phenomenon of high school graduation poses a similar situation. For instance, analysts are faced 
with the likelihood that relevant information regarding students’ enrollment status, dropout 
experiences, or kind of high school completion outcome (e.g., regular diploma, certificate, or 
GED) may be inaccurately or incompletely reported. That is, relevant aspects of the 
phenomenon will be unobserved, at least for some cases. It follows, therefore, that it will be 
difficult if not impossible to measure the rate of high school completion for a school, district, or 
state without error. As with any empirical indicator designed to produce an estimate of an 
underlying construct, the quality and accuracy of information generated regarding rates of high 
school completion will hinge on several factors, including: the operational definition adopted, 
simplifying assumptions made, and the types of data employed. 

The extant debate over the measurement of dropout and completion rates has touched 
upon these factors to some extent. In particular, authors have called attention to the ways in 
which differential treatments of certain issues (e.g., who counts as a high school graduate, data 
sources, computational methods) may introduce systematic biases into reported high school 
graduation rates (Chaplin 2002; Haney 2000; Greene 2002a, 2002b; Kaufman 2000, 2001; 
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Winglee & al. 2000). The provisions of a specific system of accountability like NGLB, 
however, impose statutory requirements and associated methodological constraints that render 
certain past areas of inquiry moot while raising the profile of other issues, at least for the 
purposes of measuring high school graduation under the current federal law. 

For instance, analysts have debated the relative merits of calculating high school 
completion rates based on data reported by youth (or their parents) residing in a particular 
geographical area (e.g., CPS or Census) as opposed to administrative data tied more directly to 
educational units like districts or schools (e.g., the Common Core of Data). An indicator based 
on geographically-based population data may be acceptable for some research and reporting 
purposes, but would likely be viewed as inappropriate for use in the context of a high-stakes 
educational accountability system. For example, an indicator capturing the percent of young 
adults who had completed high school and are currently living in the catchment area of a public 
school district would arguably be too blunt an instrument for measuring the performance and 
effectiveness of that district. This would be particularly true in areas with high private school 
attendance, migration, and residential mobility rates and where many young adults did not attend 
high school in their current place of residence. 

NCLB would appear to offer feirly clear guidance with respect to another issue that has 
been the subject of considerable debate — who should be rightfully counted as “high school 
graduates.” The main kinds of high school completion outcomes that have been incorporated in 
various indicators include: regular high school diplomas conferred to students meeting state 
requirements for high school graduation; state- issued high school credentials other than a 
diploma (e.g., issued to individuals meetings some but not all graduation requirements); and the 
General Education Development certificate, or GED. More restrictive definitions of a “high 
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school graduate” will necessarily result in a lower estimate of the completion rate by excluding 
recipients of certain credentials. The language of NCLB and in particular subsequent regulatory 
guidance indicate that only regular high school diplomas should be counted for purposes of 
accountability, while individuals receiving other state- issued credentials or the GED should not 
be considered graduates. This, however, does raise an ancillary although perhaps not 
inconsequential issue. Under om federalized system of governance it is the states rather than the 
federal government that hold the final authority to define the requirements for a high school 
diploma. As a result, the standards of performance and educational requirements attached to a 
diploma or other credential may in fact vary considerably from one state to another. This 
situation may pose a particular challenge when attempting to compare high school graduation 
rates across states, whose standards for issuing a high school diploma may differ. 

In the past, high school graduation rates have typically been calculated and reported at 
highly aggregated levels, most often appearing as estimates for the nation as a whole or in some 
cases for states or large districts. Under the terms of NCLB, however, the school represents the 
most basic educational unit for purposes of accountability. Accordingly, graduation rates must 
be calculated for each secondary school, separately for the entire student body, and for individual 
student categories defined by race and ethnicity, socioeconomic status, disability, and English 
language proficiency. Efforts to develop completion indicators at the district and state level have 
been faced with a variety of challenges, including inconsistent definitions of indicator 
components (e.g., what constitutes a “high school graduate”) and the lack of systematic data at 
the appropriate levels of aggregation. Available estimates suggest large variations in graduation 
rates across the states and districts for which information is available (Green 2002a). However, 
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the true range of values across the nation as a whole and at the school level remains largely 
unknown. 

4. Alternative Graduation Indicators 

While on the face of things developing an indicator to capture high school graduation 
rates might appear to be a relatively straightforward proposition, the discussion above suggests at 
least some of the complexities that underlie this undertaking from a methodological perspective. 
In addition, while the provisions of NCLB do provide some guidelines for calculating these rates 
for accountability purposes, a number of indicators would appear to be able to provide an 
adequate means of measuring graduation rates from a statutory perspective. Below we briefly 
discuss four distinct approaches to measuring graduation rates that might be utilized imder 
NCLB accountability. For simplicity of presentation and to maintain consistency with the 
empirical portion of this paper, we will describe the computation of graduation rate indicators for 
school districts as an exemplar in the discussions below. These four general approaches could 
also be employed to construct measures for units at other levels of the education system 
including states and schools. Indeed, the provisions of NCLB do call for performance- based 
accountability at all of these system levels. 

We have chosen to concentrate in this paper on the main features that broadly distinguish 
these approaches from one another. All of these measurement strategies introduce non-trivial 
assumptions that enable the use of a simplified functional expression for computational purposes. 
These might include speculations regarding: the quality of the data utilized for calculations; net 
stability of population size or in- and out-migration over time; and the comparability of attrition 
patterns due to dropout and transfer across grade levels and over the period of time being 
observed. Although these issues are worthy of systematic investigation, for the purposes of this 
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study we will simply remind the reader that the indicators discussed below each require that the 
analyst make similar kinds of assumptions. Later sections of the paper apply these measurement 
approaches to construct empirical indicators for the purpose of comparing their statistical 
properties. 

4.1. Longitudinal Graduation Rate 

It could be argued that, although not explicitly required, the spirit of accountability imder 
NCLB calls for student performance to be assessed by tracking individual students over time. 
This is a proposition that could apply to measuring both academic achievement and high school 
completion.'* Under this perspective the public school graduation rate for a particular student 
cohort attending a school, district, or other educational unit could be calculated as shown in 
Equation 1. For illustrative purposes in this section we will take the high school class of 1999- 
2000 as our point of reference and assume that oir-time graduation means that 9'*’ graders in 
1996-97 will complete high school four years later in 1999-2000. 



LongRate’’”'^ = 



plong 

^1999 



E - L 

^1996 ^ 1996-99 



[ 1 ] 



where: 

R[lll is the coimt of individual students from the entering 1 996 high school cohort 

who receive a regular diploma four years later in 1999-2000; 

£■, 99 ^ is the count of students enrolled in the 9**’ grade in 1996-97 (the entering high 

school cohort); and 

^ 996-99 is the count of students from the entering 1996 high school cohort who 

legitimately leave the focal school district as a result of: mobility (to another 
public school district), transfer to a private school, or death. 



Tracking individual students overtime enhances the ability to create “value-added” indicators of student 
academic performance as described by Meyer (1996). 
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Although this equation is parsimonious in form, its simplicity is only attainable through the 
implementation of a rigorous accountability system that is capable of tracking individual students 
over time and able to accurately distinguish among a number of student outcomes (e.g., diploma 
recipient, recipient of other credential, dropout, transfer from district of interest, or exit from the 
public school system altogether). In practice it may be very difficult to make such 
determinations with certainty, which raises the question of how school districts will choose to 
classify entering cohort members whose status cannot be readily ascertained four years later 
when they should be completing high school. 

In theory the use of longitudinal data may appear to offer an ideal means of measuring 
high school graduation. However, some analysts have suggested that the benefits associated 
with this methodological approach in principle may not actually materialize in practice because 
of technical hurdles involved in actually following students over time and incentives to count 
students whose status is unknown as transfers by default (Haney 2001). Such challenges may be 
particularly notable in educational systems that serve a highly- mobile student population. 

These, incidentally, are also the types of systems that will tend to have bwer completion rates 
for a variety of other reasons related to actual educational conditions rather than methodological 
considerations. For these reasons, it has been suggested that indicators based on group- level data 
from student cohorts (rather than individuals) tracked over time may actually produce more 
statistically sound estimates under certain conditions (Green 2002a). 

While the individual- level longitudinal approach to measuring graduation rates is 
important to acknowledge from a conceptual perspective, the remainder of the paper deals with 
indicators constructed from group -level cohort data. There are two reasons for this decision. 
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The first, a practical one, is that we did not have access to the systematic individual- level 
longitudinal data needed to construct such an indicator for educational systems nationwide. In 
fact, most states and districts do not maintain longitudinal databases of this kind.^ The second 
related consideration deals with the implications for policymakers. Because at present most state 
accountability systems lack comprehensive longitudinal data, the adoption of a cohort-based 
indicator is likely to be an attractive option for maintaining compliance with NCLB, at least in 
the short term. For many states this will be a necessity. 

4.2. NCES High School Completion Rate 

The National Center for Education Statistics (NCES), a branch of the U.S. Department of 
Education, annually collects extensive information on high school dropout and completion 
through the Common Core of Data. The CCD is a census of public sector state and local 
educational agencies and schools across the nation. Dropout and completion data are collected at 
both the state and district levels. In addition to its data collection efforts, NCES publishes 
widely-cited reports of dropout and completion rates for the nation as a whole and for states and 
has also sponsored technical studies examining the methodological issues involved in developing 
statistically valid estimates for these rates (see Winglee & al. 2000, Young 2002). 

The high school completion rate indicator recommended by NCES can be formulated for 
the high school class of 2000 as follows (Equation 2). 



NCESRate = 



[ 2 ] 



^ Certain states, such as California, Florida, Texas, North Carolina, and Louisiana, do have longitudinal student 
achievement data available. However, even these states may not track dropouts well and may have particular 
difficulty in determining the status of individual students who move across state boundaries. 
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where 

is the count of high school completers for the 1999-2000 school year, 

including regular high school diplomas and other credentials (but excluding 
GEDs); and 

D'j is the count of students who dropped out of grade i during the 

academic year starting in year j. 

Several distinguishing features of this indicator deserve mention. As is the case for other 
cohort- level measures, one benefit of the NCES indicator is that it does not require the tracking 
of individual students over time and therefore makes relatively modest demands on data 
collection systems. Calculating this estimate for a grade cohort requires five pieces of 
information: completion counts fi-om the reference academic year (1999) and cumulative 
dropout counts Ifom each of the past four years (1996 through 99). By comparison, calculating a 
similar estimate using the longitudinal method described above requires multiple pieces of 
information Ifom each individual student in a grade cohort. 

NCES defines as high school graduates all students who receive either a regular state- 
issued diploma or another type of completion credential. The latter category does not include 
GED certificates, which are not officially conferred by the public education system. This 
definition has been chosen in order to maximize the compatibility of rates across states, which, 
as noted earlier, are Ifee to establish substantially different requirements for the confirmation of a 
high school diploma. It is not clear whether this measurement approach, developed prior to 
authorization of NCLB, would meet standards of compliance for purposes of accountability 
under current law, which explicitly links high school graduation with receiving a regular 
diploma. 




16 



21 



Swanson & Chaplin 



Counting High School Graduates 



A final point to consider is that the NCES measurement approach relies ve ry heavily on 
dropout counts. This is because the denominator of the expression estimates the size of the 
entering 1996 high school cohort by taking the sum of: (1) graduates in 1999-2000 (the expected 
year of graduation) and (2) the total count of dropouts at the cohort’s modal grade level over the 
past four years. Two major difficulties are often encountered with dropout data. The first is that 
systematic dropout statistics are often unavailable for a substantial number of states and districts, 
which raises the possibility of bias in coverage and ftulher complicates efforts to produce state- 
or even nationally- representative estimates for high school completion rates (Young & Hoffinan 
2002; Greene 2002a). ^ Second, concerns have been raised that publicly reported dropout counts 
may substantially underestimate the true number of dropouts. We would anticipate this bias to 
be particularly severe for data generated in the context of state accountability systems, where 
there may be substantial incentives to categorize students whose status is unknown as transfers 
rather than dropouts or to take other liberties with accounting procedures in order to avoid 
sanctions associated with high rates of dropout (Haney 2001). 

4.3. The Greene Method - Cohort Graduation Rates 

Another commonly referenced cohort- level indicator for high school graduation rates has 

been developed and employed in work by Greene (2002a, 2002b). This approach estimates a 
cohort graduation rate using information on the number of high school graduates in a given year 
and the size of the 9'*’ grade cohort four years earlier. This measure represents a cohort 
completion rate and can be calculated using data from the CCD or similar sources, as shown in 
Equation 3. 

* The Common Core of Data is the most comprehensive national source for dropout information currently 
available. Even so, for the 1999-2000 school year CCD reports dropout rates for only the 36 states (and District of 
Columbia) that collect dropout data using the standard definition recommended by NCES (Young, 2002). The 
majority of the remaining states have not expressed plans to change their existing methods (Winglee et al, 2000). 
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GreeneRate = 



R 



1999 



- 9-12 



E + 

^1996 ^ 



^ 9-12 _ pV 
p9 * -^1999 -^1996 

■^1996 



- 9-12 

^1996 



[3] 



where 

R^ggg Is the couiit of regular high school diploma recipients for the 1999-2000 school 

year; 

E^ggg Is the slze of the 9'*' grade cohort in 1996-1997 

E^ggg Is tiic count of students enrolled in grades 9-12 in the 1999-2000 school year, and 

E^gg^ is the count of students enrolled in grades 9-12 in the 1996-97 school year. 



The Greene method counts only regular high school diplomas in graduation counts. 
Since it imposes a more restrictive definition of successful high school completion, we can 
anticipate that it will produce systematically lower estimated graduation rates than approaches 
that count students receiving other credentials among high school graduates. This, of course, 
assumes that all other considerations are equal, whereas it is likely that other factors also 
contribute to systematic differences in estimates across different indicators. By relying on 
enrollment counts for districts, the Greene method avoids relying heavily on data that are 
especially subject to biased reporting because they are directly implicated in accountability 
regimes (e.g., dropout counts). This measurement strategy also makes modest demands on data 
availability, requiring information only on: the number of diploma recipients for a target year, 
the count of 9‘*' graders four years earlier, and the total high school- level (grades 9-12) 
enrollment for the target and base years. 
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This indicator incorporates information on enrollment counts separated by a relatively 
wide span of time, during which a school district or school may have experienced substantial 
shifts in its enrollment base. If present and unaccounted for, these population changes could 
distort the estimated graduation rate. Greene, therefore, makes an adjustment in the enrollment 
count in the denominator of the formulation to take into account the overall trend in total high 
school enrollment over the period of observation. The actual count of 9 graders enrolled in 
1996 is adjusted (upward or downward) based on the percent change in the focal district’s total 
high school level enrollment between the 1996 and 1999 school years. 

4.4. Cumulative Promotion Index (CPI) 

Finally, as a somewhat differently oriented approach to the problem of estimating 

completion rates we propose for consideration a Cumulative Promotion Index (CPI). This 
indicator bears some resemblance to measures of holding or promoting power that have at times 
been employed at the school level, although rarely used for higher-order educational units 
(Balfanz & Legters 2001). As illustrated in Equation 4, the value of the CPI index reflects the 
probability that a student entering the 9*'’ grade will complete high school on time with a regular 
diploma. It does this by conceptualizing high school completion as a stepwise process composed 
of three grade-to-grade promotion transitions (9 to 10, 10 to 11, 1 Ito 12) in addition to the 
ultimate high school completion event (12 to graduation). 



^ To the extent that dropout rates change over the period of observation, Green’s adjustment strategy will probably 
underestimate variation in graduation rates to some degree. An increase in a district’s dropout rate between 1996 
and 1999, for example, would influence the completion rate calculations in two ways. All else being equal, an 
increased dropout rate will result in a lower number of graduates by 1999 (the numerator of the equation). The 
impact of this decrease in the numerator, however, would be partially counterbalance°d by a change in the 
denominator. Specifically, a higher dropout rate will also cause the district’s high school enrollment base to decline 
between the time the focal cohort enters 9*'' grade and its expected completion year. In turn, this would reduce the 
size of Greene’s adjustment factor for change in district enrollment (captured in the equation’s denominator). 
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where 

is the count of regular high school diploma recipients for the 1999-2000 school 
year; 



is the size of the 9'*’ grade cohort for the 1 999-2000 school year; and 
£2000 is of ^0 iO'*’ grade cohort at the start of the 2000-01 school year. 



This indicator differs from the others discussed above in several respects. As before, the 
graduation outcome for the high school class of 1999-2000 remains our point of reference. 
Unlike the earlier indicators that look back to 1996, however, the CPI effectively looks forward 
from the focal year. For instance, the first of the four promotion ratios incorporated into the CPI 
estimates the proportion of 9'*’ graders who are promoted to the next grade, using data on 9'*’ 
grade enrollment at the start of the 1999-2000 school year and lO'*’ grade enrollment at the 
beginning of the following school year (2000-01). Comparable promotion rates are also 
represented for the lO'*’ and 1 1'*’ grades. The final element of the index is the ratio of students 
who receive a high school diploma at the end of 1999-2000 to the students enrolled in the 12'*’ 
grade at the start that school year. This component represents the promotion rate or index for 
grade 12, which defines receiving a regular diploma as successful “promotion.” 

By multiplying these grade- specific promotion ratios together, the CPI estimates the 
likelihood of a 9'*’ grader from a particular district completing high school with a regular diploma 
in four years given the conditions in that district during the 1999-2000 school year. Rather than 
following a single cohort over a long period of time, the CPI embodies a synthetic cohort 
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approach by following multiple cohorts over a short period of time.® This strategy may have 
unappreciated advantages when used to construct indicators for purposes of accountability. 
Conceptually, the CPI places greater emphasis on prevailing educational conditions by 
empirically using data on grade- specific promotion rates from the target year (and beginning of 
the following school year) to estimate the performance of an educational system for that target 
year. The NCES and Greene approaches, by comparison effectively use past conditions (e.g., 
data from 1996) to estimate current system performance (e.g., graduation in 2000). The CPI’s 
measurement strategy of heavily weighting contemporary conditions may provide a more 
legitimate basis for estimating current levels of educational system performance and also for 
imposing sanctions that are experienced in the present. We would expect differences between 
estimates produced by present- versus past- weighted indicators to be most pronounced in 
situations where educational conditions have changed dramatically for better or worse over the 
years. ^ 



5. Data and M ethods 

As suggested above, the dual challenges facing efforts to develop and validate indicators 
for high school graduation rates are a lack of a comprehensive, authoritative source for 
information on high school completion combined with a lack of consensus over how to best 
define and compute such an indicator. Pursuant to the latter concern, it should be added that 
certain definitions of graduation rates may be appropriate for certain purposes or accountability 



Interestingly, Winglee et al (2000) find little difference between synthetic and true cohort estimates at the state 
level. 

* Although we have focused here on cohort -level indicators, the synthetic CPI approach to measuring completion 
rates could also be applied to individual- level longitudinal data. The benefits of CPI relative to other indicators may 
be particularly noteworthy in this case. A CPI-type longitudinal indicator, for instance, would require tracking 
individual students over a rather short span of time — from the beginning of one school year to the start of the next. 
Other longitudinal indicators, by comparison, would require that students be followed for a period four years. The 
data necessary for calculating CPI could, therefore, be obtained with substantially less effort and would place fewer 
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systems but not others. Since no established standard exists against which to test proposed 
indicators, analysts might pursue two general analytic approaches to better understand high 
school completion as an educational phenomenon and to explore alternative measurement 
strategies for developing accurate and reliable indicators. First, one might use a single 
computational approach to calculate graduation indicators for a fixed set of educational units, but 
do so using different data sources. Of interest here would be the stability of the indicators given 
the source of the data — e.g., federally-sponsored data collections like the Common Core of Data 
versus information collected and reported by state or local accountability systems. An 
alternative approach would employ a single source of data and compare indicators calculated 
according to multiple computational definitions of high school graduation rates. The current 
paper pursues the latter strategy and specifically examines the three cohort- level indicators 
described above — ^NCES, Greene, and CPI. 

The most comprehensive source of data on high school completion and dropout currently 
available is the Common Core of Data. Conducted by the U.S. Department of Education, the 
CCD is a census of public sector local educational agencies (districts) and schools for the fifty 
states, the District of Columbia, and several other non-state jurisdictions. Annual surveys of 
basic demographic and educational information at the state, district, and school levels are 
completed by staff of the respective state education agencies. Information on high school 
completion and grade- specific dropout counts are reported at the district level, whereas grade- 
specific enrollment data is available only at the school level. This has two consequences with 
respect to calculating cohort- level indicators. First, we must combine data from the CCD’s 
district and school databases. Second, because completion and dropout counts are not reported 
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for individual schools, the district is the most basic educational unit for which graduation rate 
indicators can be systematically constructed. (Grade-specific enrollment counts at the district 
level are obtained by aggregating school data.) 

During the reference year for our analysis (1999-2000), 14,978 regular school districts 
were in operation throughout the fifty states and District of Columbia. In defining our target 
population for analytic purposes, however, we will need to introduce several additional 
restrictions. Our objective is to identify districts that (1) are eligible for the calculation of a 
graduation rate and (2) should in theory have the necessary information needed to calculate such 
a rate. For instance, it is reasonable to assume that we can only calculate a meaningful 
graduation rate for districts that contain a full complement of secondary grade levels (9- 1 2). 
About 27 percent of regular school districts in the country do not meet this criterion, the majority 
of which possess only an elementary level grade span or have ungraded enrollment. In addition, 
the indicators of interest variously require pieces of information from 1996 through 2000. 
Districts not in operation during this entire period of time will be legitimately missing necessary 
information. Of the districts in operation during 1999, a small fraction (3%) were not in 
operation during at least some part of our period of observation. Further, 4.5 percent of districts 
had undergone a significant change in boundaries over this period. Such events effectively 
change the identity of a particular district and produce large year-to-year fluctuations in 
enrollments, which in turn results in invalid estimates of completion rates. Taking all of these 
criteria into consideration, we arrive at a target population of 10,836 school districts for which 
we should be able to calculate valid graduation rates. This assumes, of course, that the 
information expected to be reported for these districts in the CCD is actually present. 
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As discussed earlier, one of the main points of contention in debates over measuring high 
school graduation rates is which credentials should be counted. It is generally agreed that GED 
recipients should not be considered high school graduates for most purposes. However, a 
reasonable argument could be made in favor of counting all state- issued completion credentials 
(but not the GED) when calculating graduation rates. NCES recommends such an approach as a 
means of providing a consistent definition of “high school completer” across states. Our 
decision to count only dplomas when calculating graduation rates in this study was motivated by 
several considerations. First, this approach adheres to the definition of high school graduation 
stipulated in NCLB. Second, two of the three measurement approaches of interest recommend 
counting only diplomas (Greene, CPI). Third, only a very small percent of high school 
completers nationwide (1.5%) were awarded a non-diploma credential in 1999-2000 and this rate 
never reaches 10 percent for any individual state (Young 2002). 

6. Results 

In the following section we compare the empirical properties of three cohort-level 
indicators of high school graduation rates generated using the NCES, Greene, and CPI methods 
as described above. Our goal is to develop an understanding for the adequacy of these 
approaches for producing measures that might be used in the context of an educational 
accountability regime similar to NCLB. Among the characteristics we examine, for example, is 
the coverage the respective indicators provide (e.g., the proportion of eligible districts for which 
a valid graduation rate can be calculated). In addition to presenting results for the nation as a 
whole, state- level estimates of the high school graduation rate are also reported. In a final set of 
analyses, we disaggregate the national data to examine patterns along two dimensions or district 
characteristics that figure prominently in policy debates — the racial and ethnic minority 
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composition and the total size of the district. Graduation rates are also reported for the 100 
largest districts in the nation. 

In this paper we report estimated graduation rates for the nation as a whole and for the 
states, as well as results disaggregated by the selected district characteristics just described. It is 
important to stress, however, that the analyses presented below are largely descriptive and, while 
suggestive, should not be considered definitive. At this point, we believe it is still premature to 
either endorse or advise against using particular high school completion indicators — including 
the ones examined in this study — for use in operating accountability systems. We anticipate that 
this line of research will ultimately inform public policy debates concerning the appropriate ways 
to develop indicators that capture the performance of educational systems and organizations. 

The present investigation is best viewed as a first step, a foundation upon which to build a more 
extensive research program capable of delivering the kinds of scientifically-based evidence and 
informed policy recommendations that we believe are needed in this area of inquiry. 

6.1. National Estimates - Graduation Rates and Indicators of Indicator Quality 

The first column of Table 1 presents district- level estimates of the four-year graduation 

rate for the high school class of 2000 calculated using data from the CCD and according to the 
computational procedures described earlier. It should be noted that these initial sets of estimates 
are not weighted. These values should, therefore, be interpreted as the graduation rate for the 
average district (where large and small districts are treated equally). When using the NCES 
method, we arrive at an average district graduation rate of 85 percent. Estimates using the Green 
and CPI strategies, on the other hand, produce similar and much lower value — graduation rates 
of 75 and 73 percent respectively. As discussed earlier, the NCES measurement approach 
depends heavily on information about high school dropout to estimate a completion rate. The 
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considerably higher value found for the NCES indicator would be consistent with the proposition 
that dropout counts are substantially underreported in practice. Although more extensive 
investigations will be required to make a more definitive determination, the similarity of the 
Greene and CPI estimates (neither of which employs information on dropouts) further supports 
this speculation. 

The three panels of Figure 1 present graphs illustrating the empirical distributions of 
graduation rates calculated respectively using the NCES, Greene, and CPI indicators. The 
estimated graduation rates for the average district are also noted. Consistent with its high overall 
graduation rates, the NCES indicator displays a heavily left-skewed distribution with a large 
proportion of districts displaying very high graduation rates. The Greene and CPI indicators, by 
comparison have more normal-shaped distributions that are relatively closely centered around 
their means. 

The graduation rate for the nation as a whole is, of course, a statistic of considerable 
interest to policymakers, researchers, and the public at large. But before attempting to generate 
an estimate for the national graduation rate, it will be necessary to determine whether the 
respective indicators are adequate to that task. To that end, we will examine two basic measures 
of indicator quality: (1) the percent of eligible district for which a valid estimate is available and 
(2) the proportion of high school students in grades 9-12 nationwide attending districts with valid 



Compared to CPI, the Green measure exhibits a somewhat elevated number of districts with graduation rates 
near 100 percent. This modest spike in the distribution of the Greene measure may be a result of two specific 
procedures used in the indicator’s construction. First, districts that showed a change in enrollment (i.e., Greene’s 
adjustment factor) of over 25 percent were dropped from the analysis. We reasoned that changes of this magnitude 
were likely to be the result of either misreported data or changes in district organization not otherwise identified in 
the CCD database. Second, final values of the indicator were constrained to a range between 0 and 1. This was 
accomplished by recoding/assigning cases within a reasonable margin of error (1 to 1 . 1 ) a value of 1 .0, and then 
eliminating cases with more extreme values (greater than 1.1). Similar rules, however, were also used to constrain 
the CPI indicator’ s grade-specific promotion ratio components to a value between 0 and 1 . Further investigation of 
the source and empirical impact of this discrepancy between the Green and CIP indicators is merited in future 
research. 
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estimates. These values represent, respectively, the district and student coverage for these 
indicators. 

The middle columns of Table 1 reveal the dramatic finding that the NCES indicator can 
be calculated for only 38 percent of districts nationwide. The student coverage rate is even lower 
(29%), suggesting that large districts are disproportionately imdercovered by this indicator. The 
low coverage afforded by the NCES indicator can be largely attributed to its utilization of 
dropout counts. District dropout information could be missing from the CCD database for either 
of two reasons. The major cause relates to standards for publicly reporting dropout statistics for 
CCD. As a matter of procedure, CCD only includes dropout data provided by states that use 
definitions of dropout and methods for calculating state and district dropout counts compatible 
with NCES requirements (NCES 2002). In our target academic year (1999-2000), for example, 
14 states did not submit dropout data compatible with the CCD definitions. Several other states 
were non- reporters in at least one additional year between 1996 and 2000. Because the 
calculation of completion rates requires dropout counts from four separate years and because the 
group of non-reporter includes most of the largest states (e.g., California, Michigan, New York, 
Texas), missing data exerts a very strong impact on district and state coverage for the NCES 
indicator. The second reason that a valid district estimate using the NCES method may be 
unavailable is that individual districts (within states with compliant procedures) did not report 
their number of dropouts. This possibility will be considered in the following section of the 
paper, where graduation rates are calculated separately for each state. Re gardless of the specific 
reason for missing data, however, the very low level of population coverage makes it inadvisable 
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to attempt to compute a nationally- representative estimate of the graduation rate for the class of 
2000 using the NCES method. ' ' 

By contrast the Greene and CPI indicators each provide a very high level of coverage for 
both the district and student populations nationwide. Using the Greene method we obtain a valid 
result for 86 percent of districts and 89 percent of students across the nation, while comparable 
coverage levels for the CPI index are 85 and 93 percent respectively. For both of these measures 
(particularly the CPI), rates of coverage are somewhat higher for the student population than for 
districts. This indicates that these approaches do a good job of capturing larger districts (which 
serve larger student populations). These high levels of population coverage suggest that it 
should be possible to use these methods to produce national estimates of the high school 
graduation rate that are reasonably accurate and representative of students across the coimtry. 

We do this by weighting individual district rates to reflect relative size of their high school- level 
enrollment. We find that the resulting national estimates are cons iderably lower that those for 
the average district. Using the Greene method, we project that only 65 percent of students 
graduate from high school on time’^ with a regular diploma. A nearly identical estimate for the 
national graduation rate (66%) is obtained for the CPI indicator. The disparity between the 
district average (imweighted) and national average (weighted) results, again, reflects the 
influence of large districts where graduation rates tend to be lower. This issue will be addressed 
more directly in a later section of this paper. 



' ' It is, in fact, for this reason that the U.S. Department of Education relies upon other methods and data (e.g., the 
CPS) in its official reporting on high school completion and dropout rates. 

The estimates reported in this paper are based on completion counts that may include late (and early) graduates. 
Therefore, they do not represent the experience of just “on-time graduates” strictly speaking. All three of the 
indicators we examine all make the assumption (which cannot be tested using these data) that the number of off-time 
completers will be offset by the number of student who were retained (or accelerated) in the enrollment base. We 
expect that on average violations of this assumption will be small in magnitude. While such violations might affect 
the absolute completion rates reported, we also anticipate that they should have an approximately equal affect on all 
indicators (since they make the same assumption). 
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6.3. State Estimates - Graduation Rates and Indicators of Indicator Quality 

Tables 2 through 4 present state-by- state estimates of on-time graduation rates for the 

high school class of 2000 as produced respectively by the NCES, Greene, and CPI measurement 
approaches. Each table reports the unweighted district graduation rate (i.e., the rate for the 
average district in the state), followed by indicators of the proportion of districts and students in 
the state for which a valid rate can be calculated, and then finally a weighted estimate indicating 
the average graduation rate for students statewide. 

As noted earlier, CCD does not report district dropout counts for states whose dropout 
reporting practices do not conform to NCES definitions. As a result the NCES indicator for the 
1999-2000 completion rate (which uses dropout information from 1996 through 1999) cannot be 
calculated for 24 states or the District of Columbia. Estimates are also unavailable for two 
additional states due to missing completion or dropout data for a particular year (see Table 2). 
Among the remaining 24 states for which estimates can be computed, we find very high average 
levels of indicator coverage for both districts and the student populatioa While the within-state 
coverage levels for districts and students range as low as 67 and 83 percent respectively, the 
majority of states approach near-total coverage in both regards. Indeed, only two states report 
coverage rates below 95%. Earlier we noted that the large amount of missing data for the NCES 
indicator prevented the calculation of a national graduation rate using this measurement method. 
We can now safely conclude that the reason these data are unavailable is almost entirely 
attributable to non- conforming state procedures for collecting dropout data, rather than district- 
specific non- reporting in conforming states. The final column of Table 2 reports statewide 
graduation rates that are calculated by weighting each district’s estimate by its relative share of 
the within-state 9-12 grade student population. State graduation rates range from a low of 63 
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percent for Louisiana to a high of 90 percent for North Dakota. The majority of states, however, 
fall within a more narrow range of 75 to 85 percent. 

In Table 3, we find that the Greene method can be used to calculate graduation rates for 
districts in 48 states plus the District of Columbia. We also find that this indicator provides a 
consistently high level of coverage within these jurisdictions. On average about 89 percent of 
districts within states have a valid estimate, with only two states falling below 75 percent 
coverage (Alaska and Oklahoma). Ninety-three percent of the student population in the average 
state is represented by the Greene graduation rate, with only a single state displaying a level of 
coverage below 75 percent (Arkansas). Graduation rates for the individual states average about 
68 percent, although values range considerably around this mean. Florida displays the lowest 
graduation rate using the Greene method, with less than half (49%) of the class of 2000 
graduating fi-om high school on time. At the other extreme North Dakota graduates nearly 84 
percent of its high school students. 

Using the CPI computational strategy (Table 4), we find results very similar to those 
produced by the Greene method. With the exception of Arizona, the CPI indicator can be 
computed for all states and the District of Columbia. Levels of CPI indicator coverage for 
districts and students are consistently high, averaging 88 and 94 percent respectively. These 
values are comparable to those reported for the Greene indicator above. The average CPI 
graduation rate across states is 68 percent. Individual state values range fi-om a low of 48 percent 
for South Carolina to a high of 82 percent for New Jersey. A closer examination of the results 
produced by the Greene and CPI methods reveals a high overall level of agreement between the 
two indicators at the state level. These measures display a very high correlation (r = .92) and 
also generate very similar rankings of states on the whole. 
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6.4. Graduation Rates and District Characteristics - Minority Enrollment and Size 

In order to start developing a basic understanding of the ways in which these alternative 

indicators might (differentially) operate if put into practice, in this section we examine the 
relationship between estimated graduation rates and two district characteristics that figure 
prominently in the educational policy arena — racial and ethnic composition and district size. For 
the purposes of this study we will define a district’s minority enrollment as the percent of non- 
White (non-Hispanic) students enrolled in grades 9-12 during the 1999-2000 school year. For 
each measurement approach, Table 5 disaggregates graduation rates and district coverage 
indicators by several levels of district minority enrollment. 

All indicators display the consistent negative relationship between minority composition 
and graduation rates. Districts serving student populations with higher proportions of racial and 
ethnic minorities display consistently lower graduation rates, a finding well documented by 
earlier research. These disparities can be substantial with graduation gaps between districts with 
no minority students and those with over 75 percent minority enrollment reaching as much as 28 
percent (e.g., 84% versus 57% for the unweighted Greene indicator). Much as was observed 
earlier, we find that district coverage for the NCES indicator is ge nerally low across all minority 
enrollment strata. By contrast, the Greene and CPI methods generate valid estimates for a high 
proportion of districts at all minority enrollment levels. The graduation rates estimated by the 
NCES measure are also generally considerably liigher than those for the other est im ators. 

Table 6 disaggregates results by six district size strata. In order to maintain consistency 
with other publicly reported statistics, we define size here as total student enrollment in grades 
PK-12 during the 1999-2000 school year. Because overall district size correlates very highly 
with secondary level enrollments, it is unlikely that the results reported below would differ 
noticeably had we employed them on district enrollments for the secondary grades only. As we 
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would expect based on prior research and reporting, there is a consistent negative relationship 
between district size and graduation rates — these rates being systematically lower for larger 
districts. 

Consistent with earlier results, we find that the NCES indicator produces higher estimates 
for graduation rates at all district size levels when compared to the two other indicators. Levels 
of coverage are also quite low on average, and dramatically so for very large districts. Whereas 
valid NCES estimates are available for just over 20 percent of districts with enrollments greater 
than 25,000 students, over 90 percent of these districts are covered by the CPI indicator. 
Weighted results from the Greene and CPI methods also highlight the extent of the crisis very 
large districts face with respect to high school completion. Among the largest districts (those 
with enrollments of over 50,000) barely half of all students can be expected to graduate from 
high school on time with a regular diploma. By comparison, graduation rates in the smallest 
districts may reach as high as 80 percent on average. 

In keeping with another common reporting convention, our final analysis estimates 
graduation rates for the 100 largest school districts in the nation. Table 7 reports these values 
and ranks the results by the district’s total PK- 12 enrollment. Although we present this table 
largely for illustrative purposes, several points are worth noting. First, as we have found 
throughout this study, the NCES affords very poor coverage of large school districts. Among 
these 100 largest districts, a valid NCES estimate is available for only 27. None of the five 
largest districts are represented, and only 3 out of the top 25. By comparison, the Greene and 
CPI indicators are available for 87 and 94 of these districts respectively and both provide near- 
total coverage of the very largest districts. 
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Although the analyses above found very low average graduation rates among very large 
districts, even this group displays a surprising range of values. Taking the CPI estimates as our 
point of reference, the graduation rates estimated for individual districts range from 17.6 percent 
for Cinciimati, Ohio, to 97.6 percent for Fairfax County, Virginia. It is reasomble to expect that 
values generated for progressively smaller educational units (e.g., states vs. the nation or districts 
vs. states) are likely to produce progressively more unstable estimates. Values for the 100 
largest districts generated using the Greene and CPI methods do show a large degree of 
agreement (r = .86). However, there are a number of cases in which substantial differences 
occur. To take one example, the estimated graduation rate for Boston diverges by 20 percentage 
points (60% for Greene and 80% for CPI). Although a systematic consideration of these issues 
is beyond the scope of the current study, the reasons underlying large differences in the 
performance of the alternative indicators merit careful examination in future research. 

7. Discussion and Conclusions 

One of the objectives of this paper has been to engage in a thoughtful consideration of the 
issues attendant to the measurement of an important indicator of educational system 
effectiveness — high school graduation rates — within the context of performance based 
accountability regimes. In particular we have been concerned with the implications of different 
measurement approaches under the recently authorized and hotly-debated No Child Left Behind 
Act. To this end, we engaged in a brief review of the relevant law and policy guidance and 
attempted to explicate in at least some small amount of detail several major empirical approaches 
that states may opt to pursue under NCLB. It is our hope that these discussions have been 
informative in highlighting some of the issues that are likely to be of major political and material 
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consequence in the months and years to come as states continue to implement the new federal 
statutes. 

The empirical aspects of this investigation have, admittedly, been largely exploratory in 
character and are best viewed as illustrative at this stage. Nevertheless, several important 
preliminary lessons can be gleaned from this work and fruitfully applied in future research along 
similar lines. We briefly summarize. 

The measurement of educational performance is a major component of accountability 
under the terms of No Child Left Behind. The majority of public attention in this regard has 
been devoted to the task of implementing state mandated student assessment programs and the 
associated technical, fiscal, and political challenges. However, high school graduation is another 
essential component of educational performance under the new accountability regime, and one 
that in some respects is intended to serve an important check with regard to achievement. 

Systems that may be tempted to boost achievement scores by eliminating low-performing 
students from the rolls will be able to do so only to the detriment of their graduation rates. At 
least that is how the system is supposed to operate. States or districts that continue to avoid 
devoting serious attention to the way they define and measure graduation rates may eventually 
find themselves in difficult straits with regard to attaining compliance with the new law. As this 
paper has shown, calculating an apparently- simple value — the percent of students who graduate 
from high school — is anything but simple and the best way to go about it is anything but 
apparent. 

Many otherwise well- informed parties generally hold the view that we are living in a 
golden age of educational attainment. More students than ever are attaining more years of 
education and the days in which large numbers of youth end their education before finishing high 
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school are largely a thing of the past. To a certain extent this is true. However, recent research 
has begun to suggest, and our results further confirm, that graduation rates are very likely to be 
much lower than the 85 to 90 percent that prevails in the conventional wisdom. At least with 
respect to four- year completion with a regular diploma, a growing body of work is pointing to a 
graduation rate as low as 65 percent nationwide. Even that statistic, which would strike many of 
us as surprisingly and unacceptably low, overlooks truly abysmal performance that prevails in 
certain places. In very large districts, in those educating large numbers of disadvantaged 
minority students, and in states with historically struggling educational systems, the odds of 
graduating fi-om high school for the average student lie well below 50:50. 

Although the empirical results presented above should be interpreted cautiously, one 
clear and consistent finding that emerges fi-om all of our analyses is that graduation rates based 
on dropout counts are highly problematic. And alternative definitional and computational 
strategies should be seriously explored. The root of this problem lies in the wholly inadequate 
state of our national systems for defining and collecting information about dropping out. It may, 
in fact, be more appropriate to describe this system as non-existent for most practical intents and 
purposes. Even in the CCD, the nation’s census of educational agencies and schools and most 
comprehensive source of basic educational statistics, dropout data goes unreported for large 
numbers of states and districts, and the data that are available greatly understate the likely 
magnitude of the dropout problem. 

The fact that the largest of the states number among the non- reporters exacerbates the 
poor level of coverage that dropout-based indicators of graduation provide for the student 
population. To further complicate matters, many of these same states are also home to large and 
rapidly growing minority populations. The challenges associated with educating these groups 
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are of paramount importance to policymakers nationwide. In fact, concerns that the educational 
needs of these groups may be going unmet played a large role in motivating NCLB’s 
requirement that performance gains be demonstrated for all students, particularly those from 
racial and ethnic minority groups and those with limited English proficiency. 

But even in situations where presumably valid dropout counts are available and can be 
used to calculate high school graduation rates, the estimates that incorporate such information 
appear to be implausibly low in many cases. That is, at least in comparison to rates based on 
enrollment counts. It is clearly imperative that we develop more accurate information on the true 
dropout rates that prevail in schools and districts across the country. There is some reason to 
hope that the stronger and more systematic forms of accountability that will accompany NCLB 
may promote some progress in that very direction. But in the meantime, the best evidence 
available clearly recommends a move, and a rapid one at that, away from reliance on graduation 
rates using dropout data and towards alternative strategies for measuring this key indicator of 
educational performance. 
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